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Abstract 

We further evaluated the Virtual Lecture Hall (VLH) ( Cramer, Collins, Snider, & Fawcett, 
in press), an instructional computer-based plalform to deliver PowerPoint slides threaded with 
audio clips for later review. Students from either an in-class or online section (ns = 810 and 
74 respectively) of introductory psychology had access to live recorded lectures via the VLH, 
made available through the course Web site. Approximately 45% of in-class and 78% of 
online students used the resource prior to each of two course midterms; 32% of in-class and 
50% of online students completed a five-item survey assessing student perceptions of whether 
the VLH enhanced learning or increased grades, and whether they wanted the resource in 
other courses. Number of VLH accesses and total duration were calculated. Residts showed 
that regardless of course section, greater VLH use was linked to higher midterm scores, and 
student perceptions of the VLH were highly favorable. Curiously, whereas in-class sttidents’ 
VLH use and duration were negatively related to expected grade, that same link was positive 
for online students. Directions for fiiture research in resource development and implications 
for educators are discussed. (Keywords: PowerPoint-audio recorded lectures.) 

The face of the classroom has changed dramatically in recent years, and con- 
tinues to do so in higher education (Carswell, Thomas, Petre, Price, & Rich- 
ards, 2000; Clark & Mayer, 2002; Comeaux, 2005; Mayer, 2004; Nicolaou, 
Nicolaidou, & Constantinou, 2005; Zhang, 1998). Bransford, Brown, and 
Cocking (2000) noted that “new developments in the science of learning raise 
important questions about the design of learning environments - questions that 
suggest the value of rethinking what is taught, how it is taught, and how it is 
assessed” (p. 131). As Alonso, Topez, Manrique, and Vines (2005) observed, 
“Computer-assisted teaching using the Internet has radically changed the teach- 
ing paradigm” (p. 217). In fact, more students than ever are taking distance ed- 
ucation courses, and more instructors are utilizing technology in the classroom 
(Kekkonen-Moneta & Moneta, 2002). 

Whereas instructors routinely entertain the introduction of novel technolo- 
gies in the classroom in order to stimulate student interest, focus attention, and 
foster learning, there remain many unique challenges (Barrie, Ginns, & Prosser, 
2005; Comeaux, 2005; Finley, 2005; Palloff & Pratt, 1999). Although often 
confronted by practical student constraints, including time, finances, geogra- 
phy, and technology, online instructors have successfully revamped their online 
learning environment (a) to increase learning availability to those who cannot 
or choose not to attend traditional in-class offerings, (b) to cost-effectively de- 
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sign and disseminate course content, and (c) to handle larger student numbers. 
For example, various online tools such as course-relevant newsgroups, threaded 
discussion boards, even student self-assessment strategies help to encourage 
critical thinking and effective writing skills. 

However, the question remains as to whether the advent of educational tech- 
nologies, including PowerPoint slides, online discussions, and computerized 
demonstrations, live up to students’ expectation of truly enhanced learning for 
which they were intended (Barrie et ah, 2005; Bork, 2001; Frear & Hirschbuhl, 
1999; Herson, Sosabowski, & Lloyd, 1999; Sosabowski, Herson, Lloyd, & 

Bell, 1998). Indeed, the advent of the Internet in recent years has brought with 
it new possibilities for instructors to creatively deliver effective course instruc- 
tion to students (Carswell et al., 2000). For instance, Kekkonen-Moneta and 
Moneta (2002; see also Xie & Zhang, 1999) compared students’ learning out- 
comes in the lecture to an online version of an introductory computing course. 
Whereas both lecture and online-learning students achieved comparable factual 
learning outcomes, the online students outperformed the lecture students in 
applied-conceptual learning. Among others, these findings suggest that the use 
of carefully designed interactive learning modules can effectively foster higher- 
order learning outcomes. Indeed, Martin and Taylor (1997) have regarded a 
virtual classroom as the next step in the development of educational technology. 

THE VIRTUAL LECTURE HALL 

As a bridge between these educational trends, we further extend our previ- 
ous research on the Virtual Lecture Hall (VLH) (Cramer et al., in press), a 
computer-based instructional delivery platform that marries PowerPoint slides 
with live-recorded audio clips that students may review later. It was our hope 
that students equipped with this pedagogical tool would show greater retention 
of course material and enhanced course satisfaction. Specifically, we sought to 
compare the utilization rates and performance levels between students enrolled 
in the same course, taken either traditionally in the classroom or delivered en- 
tirely online through the VLH. 

The VLH began as a platform to deliver previously presented lectures (record- 
ed either live or otherwise) so that students could review material (a) that they 
had missed, or (b) for study purposes. Compiled lectures that unite audio clips 
with PowerPoint slides are made available to students on the campus network 
and over the Internet, although lectures can be similarly saved and delivered on 
a CD-ROM. The main Internet page includes both a media player and verti- 
cal menu bar that lists the titles of slides in the lecture. Students can navigate 
directly to any slide in the lecture that they wish to review by merely selecting a 
slide title that then displays the slide and plays the audio of the associated slide. 

For the most recent incarnation of the VLH, a PowerPoint add-in was devel- 
oped that provided easy-to-use controls for voice recording and synchroniza- 
tion with slides on the computer as the lecture was presented. The add-in both 
handled post-lecture production chores and published the finished lecture to 
a streaming RealNetworks media server. Two versions of each lecture were cre- 
ated, as most students had telephone dial-up connections of 33.3 or 56.6 kB/s. 
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One version was created using D-HTML for students with fast Internet con- 
nectivity (DSL or cable). The D-HTML version allowed VLH users to adjust 
the size of the on-screen slides for easier viewing while still maintaining slide 
animations and correct audio timings. The other version accommodated stu- 
dents with slower telephone modem connectivity. In this version, audio was 
synchronized with a JPEG image of each lecture slide. This new system also pre- 
served the capability of hyperlinking to any slide in the presentation. 

In the first empirical study of this technology, Cramer et al. (in press) made 
the VLH available (after an initial baseline of student performance) to more 
than 800 undergraduates enrolled in either Introductory or Personality Psychol- 
ogy, and both utilization and duration statistics were collected. With test perfor- 
mance assessed on the first midterm prior to availability of the VLH, students 
served as their own control group so any initial differences could be covaried 
or partialled out of subsequent test performance (i.e., the second midterm). In 
addition to utilization data, students elected to complete a brief survey assess- 
ing whether they believed the VLH enhanced learning or increased grades, and 
whether students wanted the VLH in other university classes, how many classes 
they have missed to date, and their expected grade. Results showed a positive re- 
lation between subsequent test performance and both the number and duration 
of VLH accesses, even after accounting for students’ initial midterm scores. In 
addition, there was a significant difference between high- and low-users of the 
VLH. In other words, students who used the VLH a total of 100 minutes or 
more scored approximately 15% higher on their second midterm than students 
who used it less. Analysis of the student survey showed a highly favorable evalu- 
ation of the VLH, wherein a majority of students believed it would enhance 
learning and improve grades. More than 90% agreed the VLH should be of- 
fered in other courses. Results failed to find a relationship between VLH utiliza- 
tion and the number of classes missed, or between expected grade and either 
VLH access or duration of use. 

PRESENT STUDY AND HYPOTHESES 

At present, the VLH has been evaluated on students who attend in-class 
lectures and may revisit recorded lectures for review for a later time. That is, it 
remains unknown how effective would be the VLH when delivered to students 
registered for the same course with lectures delivered entirely online. As such, 
we evaluated the performance of students participating in an in-class section of 
Introductory Psychology (given the option of using the VLH), and compared 
them to students enrolled in an online section of the same course whose VLH 
access to course lectures was necessary. This permitted the evaluation of two im- 
portant hypotheses. 

Based on earlier work by Herson et al. (1999), who tracked student utiliza- 
tion rates, it would prove worthwhile to monitor who uses the VLH according 
to how often they use the feature and for how long. Thus, both a measure of 
number and duration of accesses were calculated. Upon entry to the VLH on 
the course Web site, the server recorded each time a student accessed a lecture, 
logging the date and time of each entry as well as the duration of each access. 
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Students were also asked to complete a survey delineating their thoughts and 
feelings about the VLH, including their perceptions of its effectiveness, whether 
they believed it would increase their grade, and whether they would like to see 
the VLH employed in other classes (cf. Carswell et al., 2000; Cramer et ah, in 
press; Johnson, Aragon, Shaik, & Palma-Rivas, 2000). We hypothesized that 
compared to their performance assessed before access to the VLH, students who 
use the VLH more often or for a greater duration will show significant improve- 
ments in their subsequent test scores. We further hypothesized that because this 
is the only means by which online students can receive the lecture materials (un- 
like in-class students receiving a live lecture), the aforementioned improvements 
should be significantly stronger for students enrolled in the online section. 

METHOD 

Participants 

There were 884 students from an introductory psychology class (71% female) 
at the University of Windsor in southwestern Ontario, Canada, who partici- 
pated in the study for no compensation. Seventy-four of these students (8.4%) 
were registered in an online section without on-site classes; the remaining stu- 
dents were registered in two relatively evenly populated on-site in-class sections 
of the same course with the same instructor, materials, and evaluation. 

Materials and the VLH Platform 

The VLH lectures were created using PowerPoint slides to deliver the static 
lecture material. The audio component was recorded simultaneously during 
their presentation to the in-class sections. The compiled lectures were uploaded 
and made available to students through a course Web site open only to students 
registered in the introductory course. Prior to gaining access to the resource, 
students provided informed consent (by endorsing through a mouse click an 
online agreement) in their use of this educational tool, and received a disclaimer 
(with each VLH access) that their activity — as tracked through a student num- 
ber — would be monitored for research purposes. 

In addition to utilization data, students were invited to complete a brief five- 
item survey to assess their thoughts and feelings about the VLH, including ques- 
tions concerning whether they believed the VLH enhanced learning or increased 
grades, and whether students wanted the VLH in other university classes (each 
question was assessed on a five-point Likert scale: l=“not at all” to 5=“to a great 
extent”), how many classes they have missed to date (0, 1-3, 4-6, 7-9, or 10 or 
more), and finally their expected grade in the class (A, B, C, D, or F). 

Procedure 

Beginning in the Fall 2003 semester, two 50-minute lectures were recorded 
live each week throughout the semester and made available online approxi- 
mately 15 minutes after each class. As delivered with chiefly PowerPoint slides 
and video clips, the class lectures only moderately overlapped with textbook in- 
formation, so that the former reinforced (rather than substituted for) the latter. 
Five weeks after the start of classes, all students completed their first midterm, 
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followed four weeks later by completion of their second midterm. At the course 
conclusion, all students were debriefed on the study in a written memo that 
outlined all the details on why and how the study was conducted, what the re- 
searchers expected to find, and the implications for higher education. 

RESULTS 

In-Class vs. Online VLH Utilization 

In addition to utilization rates and duration measures, the analysis to follow 
tested the two hypotheses in a comparison of in-class and online students’ per- 
formance scores for the two midterms. Of the 810 in-class students, 357 (45% 
of in-class enrollment) utilized the VLH a total of 1,499 times (at five minutes 
minimum per access). There were on average 4.08 ( SD = 3.91) accesses per 
in-class student, which ranged from one to 32 in total accesses. VLH duration 
for in-class students ranged from five to 299 minutes, with a median of 16 min- 
utes. Comparatively, there were 58 of 74 online students (78% of enrollment) 
who utilized the VLH a total of 467 times. There were on average 8.05 (SD 
= 7.30) accesses per online student, which ranged from one to 37 in total ac- 
cesses. VLH duration for online students ranged from five to 365 minutes, with 
a median of 61.2 minutes. 

Tracking Midterm Test Scores 

Students in both in-class and online sections were simultaneously tested on 
their mastery of course material based on two midterms; we compared whether 
midterm scores varied by section, but more important, we explored the extent 
to which students had used the VLH in their preparation. An analysis of vari- 
ance (ANOVA) was conducted with the first midterm as the dependent variable 
and both course section (in-class vs. online) and VLH use (use vs. no use) as the 
independent variables. Results showed two significant main effects by (1) sec- 
tion, F( 1, 880) = 30.62,/) < .001, MSE=3 08, so that in-class students outper- 
formed online students (Ms = 63.0 vs. 51.9, SDs = 16.2 vs. 20.5, respectively); 
and (2) VLH use, ,F(1, 880) = 4.05 , p = .045, MSE = 308, so that students 
who used the VLH outperformed students who did not (Ms = 62.8 vs. 61.4, 
SDs = 17.0 vs. 16.7, respectively). There was no significant interaction. 

A similar ANOVA utilized the second midterm scores. As before, results 
showed two significant main effects by (1) section, F ( 1, 880) = 5.07,/) = .025, 
MSE = 409, so that in-class students outperformed online students (Ms = 60.2 
vs. 57.4, SDs = 20.4 vs. 20.0, respectively); and (2) VLH use, ,F(1, 880) = 8.77, 
p = .003, MSE = 409, so that students who used the VLH outperformed stu- 
dents who did not (Ms = 62.5 vs. 57.7, SDs = 20.5 vs. 20.0, respectively). Once 
again, there was no significant interaction. 

To further delineate the relation between midterm performance and VLH 
access, we designated VLH access into five groups: no accesses (n = 459), 1-2 
accesses (n = 184), 3-4 accesses (n = 80), 5-6 accesses (n = 68), and 7 or more 
accesses (n = 93). An ANOVA of second midterm scores by section and VLH 
access showed a significant main effect by section (as previously reported), but 
also a significant main effect for access, F (4, 874) = 3.51 , p = .007, MSE = 408. 
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Tukey follow-up multiple comparison procedures showed that students who 
accessed the VLH either 1-2 times or not at all (Ms = 55.8 and 52.8, respec- 
tively) had significantly lower midterm scores than students who accessed the 
VLH 3-4, 5-6, or 7 or more times (Ms = 62.5, 61.5, and 64.4, respectively), 
whose means did not significantly differ. That is, moderate use of the VLH cor- 
responded to a roughly 9% increase in midterm score, regardless of enrollment 
in an in-class or online section. 

Student Perceptions 

Of those who used the VLH at least once, 116 in-class and 29 online students 
(32% and 39%, respectively) completed the five-item survey. (See Table 1 for 
frequencies by section.) Overall, students from both in-class and online sections 
were especially favorable about the VLH, with students believing this tool en- 
hances learning, improves grades, and should be an option in other courses. To 
see if VLH utilization varied as a function of expected grade and class section, 
an ANOVA used both number and duration of VLH accesses as the dependent 
variables. (See Table 2 for sectional means by expected grade.) For the number of 
accesses, results showed a main effect for section, F ( 1, 138) = 1 1 .69, /> < .001, 
MSE = 27.1, so that online students on average used the VLH more than in-class 
students; and a significant interaction between section and expected grade, F (3, 
138) = 2.86, p = .039, MSE = 27.1, so that for in-class students VLH accesses in- 
creased as expected grades decreased; however for online students both VLH ac- 
cesses and expected grades increased. For the duration of accesses, results similar- 
ly showed a main effect for section, F ( 1, 138) = 14.08, p < .001, MSE = 4584, 
so that online students used the VLH for longer periods of time than in-class 
students; and a significant interaction between section and expected grade, F (3, 
138) = 3.41, p = .019, MSE = 4584; so that for in-class students duration in- 
creased as expected grade decreased, while for online students both duration and 
expected grade increased. Although the standard deviations were noteably dispa- 
rate, a reanalysis using a nonparametric alternative revealed comparable results. 

DISCUSSION 

This study compared the relative utilization, midterm performance, and per- 
ceived benefits of the VLH for students enrolled in either an in-class or online 
section of introductory psychology. Overall, results showed relatively low VLH 
utilization in the in-class section, presumably because these students could 
derive the lecture material from merely attending class. However, for online 
students, the VLH was the only means toward acquiring lecture materials and 
so their utilization rates were understandably much higher. Although relatively 
few in-class students had used the VLH when available, it was admittedly a new 
technology to both the course and the university, and some students may have 
been unsure about how to access it. Indeed, educators should not be surprised 
that although this particular tool — among others offered in any given course 
(e.g., study guides, discussion groups, useful Web links) — was made available 
to improve student performance, most students failed to take advantage of its 
availability (Karabenick, 2004). Both medical and psychological health profes- 
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Table 1: In-Class and Online Student Perceptions of the Virtual Lecture 






Hall 











Neither 







Strongly 


Agree nor 


Strongly 

Survev Item 

Sec: 

Mean 

(SD) 

Disagree 

Disagree 

Disagree 

Agree 

Agree 

Does the VLH 

IC: 

4.35 

(0.71) 

0.0% 

0.0% 

13.8% 

37.1% 

49.1% 

enhance learning? 

OL: 

3.87 

(L14) 

6.7% 

3.3% 

20.0% 

36.7% 

33.3% 

Will the VLH 

IC: 

3.93 

(0.89) 

0.9% 

1.7% 

32.8% 

32.8% 

1.9% 

improve your grade? 

OL: 

3.80 

(1.10) 

6.7% 

3.3% 

20.0% 

43.3% 

6.7% 

Should the VLH be 

IC: 

4.87 

(0.39) 

0.0% 

0.0% 

1.7% 

9.5% 

8.8% 

in other classes? 

OL: 

4.27 

(1.28) 

6.7% 

10.0% 

0.0% 

16.7% 

6.7% 





A 

B 

C 

D 

F 

Expected grade 

IC: 

— 

— 

14.7% 

51.7% 

30.2% 

3.4% 

0.0% 

in the course 

OL: 

— 

— 

13.8% 

55.2% 

20.7% 

10.3% 

0.0% 





0 

1-3 

4-6 

7-9 

10+ 

Number of classes 

IC: 

— 

— 

47.4% 

37.9% 

11.2% 

1.7% 

1.7% 

missed 

OL: 

— 

— 

n/a 

n/a 

n/a 

n/a 

n/a 


Note. IC (in-class, n = 367); OL (on-line, n = 58). 


Table 2: Mean (Standard Deviations) Number and Duration of 
VLH Accesses by Section and Expected Grade 

Expected Number of Accesses Duration of Access 

Grade In-Class Online In-Class Online 

A 3.35 (2.87) 14.50 (15.6) 26.6 (26.4) 190.2 (134) 

B 4.65 (4.83) 10.50 (7.81) 44.8 (63.2) 100.2 (100) 

C 4.54 (3.12) 7.33 (3.67) 46.4 (61.7) 101.6(67.9) 

D 7.75 (4.92) 6.50 (7.231 84.4 (92.11 64.9 (31.3) 

sionals refer to this as the service gap, wherein individuals fail to utilize the 
available resources shown to improve their situation (Cowling, Luk, Mileshkin, 
& Birleson, 2004). 

In addition, the significantly higher midterm scores observed in the present 
study for in-class students was likely due to a selection bias, wherein students 
unable to enroll in the in-class section (based on enrollment limits) elected in- 
stead for the online alternative. More generally, for both in-class and online stu- 
dents, midterm scores were significantly higher (by almost 9%) with moderate 
(three or more) accesses of the VLH; in other words, students need not immerse 
themselves in this new resource in order to derive significant and sizeable ben- 
efits. This improvement was lower than what has been observed in earlier stud- 
ies of the VLH (cf. 15%, Cramer et al., in press). It may be argued that the im- 
provement in scores from one midterm to the next is an artifact and has little to 
do with the Virtual Lecture Hall; that is, those who performed well on the first 
midterm would have similarly performed well on the second midterm, regard- 
less of access to this new technology. However, this explanation was discounted 
in initial studies of the VLH (Cramer et al., in press), wherein students acted as 
their own control groups in a pre- and post- VLH course setup that showed stu- 
dent motivation and study habits did not interact with VLH use. 
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Students in both sections were also keenly favorable toward the VLH, judg- 
ing it both to enhance their learning and improve their course grade. Not 
surprisingly, an overwhelming majority of students also wished to see the VLH 
incorporated in other courses. Indeed, although the VLH has since been used 
in both French and business classes at this university, no large-scale testing has 
been implemented to assess its relative effectiveness therein. Whether for those 
disciplines or others (e.g., chemistry, English, history, math, philosophy, phys- 
ics), it would be worthwhile to assess not only its effectiveness in the classroom, 
but also students’ perceptions of VLH as a learning tool. Certainly there may 
be a particular type of student who (more so than other students) may embrace 
this technology as a worthwhile tool. Very likely this student would be better 
skilled and knowledgeable about recent computer technology and software. 
Future research would do well to investigate the type of student who uses the 
technology. 

One novel finding in the present study involved the interaction between 
grade expectations and both VLH access and duration by section. Whereas in- 
class students expected lower course grades with greater VLH use, the opposite 
was true for online students. In other words, both the struggling in-class stu- 
dents and the superior online students were using the VLH more and for longer 
periods of time. This reinforces the notion that a particular type of student may 
be more likely to enroll in an in-class vs. online section of a course, and then 
uniquely benefit from utilization of this resource. 

We invite future researchers to consider these significant questions in follow- 
up studies: (1) Why exactly do students use the VLH? Is it for study purposes 
or rather to merely fill in the details from missed classes? Furthermore, how ex- 
actly do students use the audio clips to study — do they take notes from them or 
use them to augment notes taken earlier during the live lecture? Indeed, might 
we ask whether online and in-class students utilize the audio clips in a similar 
way? (2) How can we enhance the benefit of this feature for both in-class and 
online students? (3) Do students believe they have studied sufficiently well for 
their midterm from merely listening to the lectures? Are we creating a false 
sense of mastering the material by engaging in a potentially passive learning 
tool? (4) What particular areas of the midterm are especially affected by infor- 
mation in the audio and PowerPoint slides (i.e., what is the specific correspon- 
dence between these instructional elements)? 

As a final note, several technological improvements may be suggested for the 
VLH that may well be seen in future development and testing. At present, the 
Centre for Flexible Learning has recently improved VLH performance via the 
Internet, along with cross-platform compatibility wherein it converts narrated 
PowerPoint lectures to much smaller and more feasibly downloaded Shockwave 
movies. Preliminary work has already begun, and there presently exists at least 
one commercial product that makes this type of conversion possible. Our cur- 
rent challenge then is to smoothly automate the conversion and publishing 
process, which would give faculty a tool to allow them to publish lectures easily 
for student access. 
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APPENDIX 


Survey Instrument 

Use the 5-point Likert scale below to address the following three questions: 

Strongly Neither Agree Strongly 

Disagree Disagree Nor Disagree Agree Agree 

1. To what extent do you believe the Virtual Lecture Hall enhanced your 
learning of the course lecture material? 

2. To what extent do you believe the Virtual Lecture Hall helped to increase 
your grade in the course? 

3. To what extent would you like to see the Virtual Lecture Hall used in 
other university classes? 

Now answer the following two questions: 

4. What is your expected grade in the course? 

A B C D F 

5. How many classes they have missed to date 

0 1-3 4-6 7-9 10 or more 
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